Edge detection signal processing

ABSTRACT

To satisfactorily detect an edge detection signal of a high frequency band from a captured image signal at all times. 
     A filtering unit extracts an edge detection signal of a high frequency band from an image signal obtained from imaging, and a band control unit controls the high frequency band on the basis of lens information. For example, the filtering unit includes a first high-pass filter with a first cutoff frequency, a second high-pass filter with a second cutoff frequency that is lower than the first cutoff frequency, and an α blending unit that performs α blending on output of the first high-pass filter and output of the second high-pass filter. Even if the frequency of the edge detection signal included in the captured image signal varies due to a change in a zoom position, a lens model number, an F value or the like, the edge detection signal can be satisfactorily detected at all times.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/690,497, filed Nov. 21, 2019, which is a continuation of U.S.application Ser. No. 15/554,759, filed Aug. 31, 2017, which is aNational Stage of International Application No. PCT/JP2016/060976, filedApr. 1, 2016, and claims priority to Japanese Application No.2015-078632, filed Apr. 7, 2015. The entire contents of each of whichare incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an image signal processing device, animage signal processing method, and an imaging device, and particularlyto an image signal processing device and the like for extracting an edgedetection signal of a high frequency band from a captured image signal.

BACKGROUND ART

General cameras (imaging devices) include viewfinders (display devices)for checking the composition and focus of captured images in real time.It is required with respect to captured images displayed on viewfindersto fully reproduce the angles of view and resolution of the imagesdepending on purposes thereof. However, there are many cases, forexample, in which viewfinders with HD resolution or the like which islower than the resolution of captured images are used in high-resolutionimaging devices (e.g., 4K cameras) for the reason of convenience inhandling and installation.

The present applicant has proposed a technology for displaying edgeinformation of a high-resolution image on a low-resolution displaydevice before (refer to Patent Literature 1). This technology intends toenable focusing to be checked from edge information of a higherfrequency band than a Nyquist frequency of low resolution bydown-converting high frequency edge information, which is obtained byperforming a filtering process on a high-resolution image, anddisplaying the edge information on the low-resolution display device.

CITATION LIST Patent Literature

Patent Literature 1: JP 2014-230176A

DISCLOSURE OF INVENTION Technical Problem

Frequency bands that a high-resolution image can express significantlychange depending on a lens state (a lens model number, a zoom position,or an F number). For this reason, when frequency bands are fixed toextract high frequency edge information from a high-resolution image, itmay be difficult to obtain the high frequency edge information dependingon a lens state.

An objective of the present technology is to enable an edge detectionsignal of a high frequency band to be easily detected from a capturedimage signal at all times.

Solution to Problem

A concept of the present technology is

an image signal processing device including:

a filtering unit configured to extract an edge detection signal of ahigh frequency band from an image signal obtained from imaging; and

a band control unit configured to control the high frequency band on thebasis of lens information.

In the present technology, the filtering unit extracts the edgedetection signal of the high frequency band from the image signalobtained from imaging. Then, the band control unit controls the highfrequency band on the basis of the lens information. The lensinformation may at least include, for example, zoom positioninformation.

For example, the filtering unit may include a first high-pass filterwith a first cutoff frequency, a second high-pass filter with a secondcutoff frequency that is lower than the first cutoff frequency, and an αblending unit that performs α blending on output of the first high-passfilter and output of the second high-pass filter. In this case, forexample, the band control unit may control at least an α value of the αblending on the basis of the lens information.

In the present technology described above, the high frequency band forextracting the edge detection signal from a captured image signal iscontrolled on the basis of the lens information. Thus, even if thefrequency of the edge detection signal included in the captured imagesignal varies due to a change in a zoom position, a lens model number,an F value or the like, the edge detection signal can be satisfactorilydetected at all times.

Note that, according to the present technology, for example, the imagesignal processing device may further include a gain control unitconfigured to control a gain of the edge detection signal extracted bythe filtering unit on the basis of the lens information. Even if thegain of the extracted edge detection signal varies due to a change in azoom position, a lens model number, an F value or the like, the gain ofthe edge detection signal can be stabilized by controlling the gain onthe basis of the lens information as described above.

In addition, according to the present technology, for example, the imagesignal processing device may further include: a coring unit configuredto reduce noise included in the edge detection signal extracted by thefiltering unit; and a coring level control unit configured to control acoring level of the coring unit on the basis of imaging information,correction information, and gain information. A noise component includedin the extracted edge detection signal can be effectively reduced bycontrolling a coring level on the basis of imaging information, thecorrection information, and gain information as described above.

In addition, another concept of the present technology is an imagesignal processing method including:

a step of performing a down-conversion process on an image signal with afirst resolution obtained from imaging and generating an image signalwith a second resolution that is lower than the first resolution;

a step of extracting an edge detection signal of a high frequency bandfrom the image signal with the first resolution;

a step of controlling the high frequency band on the basis of lensinformation;

a step of performing a down-conversion process on the extracted edgedetection signal and obtaining an edge detection signal with the secondresolution; and

a step of combining the obtained edge detection signal with thegenerated image signal with the second resolution and obtaining an imagesignal with the second resolution for display.

In the present technology, the down-converting unit performs thedown-conversion process on the image signal with the first resolutionobtained from imaging and thus generates the image signal with thesecond resolution that is lower than the first resolution. For example,the first resolution may be 4K resolution and the second resolution maybe HD resolution.

The filtering unit extracts the edge detection signal of the highfrequency band from the image signal with the first resolution. Then,the band control unit controls the high frequency band on the basis ofthe lens information. The lens information may at least include, forexample, zoom position information.

For example, the filtering unit may include a first high-pass filterwith a first cutoff frequency, a second high-pass filter with a secondcutoff frequency that is lower than the first cutoff frequency, and an αblending unit that performs α blending on output of the first high-passfilter and output of the second high-pass filter. In this case, forexample, the band control unit may control at least an α value of the αblending on the basis of the lens information.

The high frequency band edge detection unit performs the down-conversionprocess on the edge detection signal extracted by the filtering unit andthereby obtains the edge detection signal with the second resolution.Then, the combining unit combines the edge detection signal obtained bythe high frequency band edge detection unit with the image signal withthe second resolution generated by the down-converting unit and therebyobtains an image signal with the second resolution for display.

In the present technology described above, the high frequency band forextracting the edge detection signal from the image signal with thefirst resolution as a captured image signal is controlled on the basisof lens information. Thus, even if a frequency of the edge detectionsignal included in the image signal with the first resolution varies dueto a change in a zooming magnification, a lens model number, an F value,or the like, the edge detection signal can be satisfactorily detected atall times, and thus edge information can be satisfactorily displayedwith high resolution on a display device, and the focus of ahigh-resolution camera can be accurately adjusted using the display of alow-resolution viewfinder.

In addition, another concept of the present technology is an imagingdevice including:

an imaging unit configured to obtain an imaging signal with a firstresolution; and

an image signal processing unit configured to process the image signalwith the first resolution obtained by the imaging unit and obtain animage signal with a second resolution for viewfinder display, which islower than the first resolution.

The image signal processing unit includes

-   -   a down-converting unit configured to perform a down-conversion        process on an image signal with the first resolution and        generate an image signal with the second resolution that is        lower than the first resolution,    -   a filtering unit configured to extract an edge detection signal        of a high frequency band from the image signal with the first        resolution,    -   a band control unit configured to control the high frequency        band on the basis of lens information,    -   a high frequency band edge detection unit configured to perform        a down-conversion process on an edge detection signal extracted        by the filtering unit and obtain an edge detection signal with        the second resolution, and    -   a combining unit configured to combine the edge detection signal        obtained by the high frequency band edge detection unit with the        image signal with the second resolution generated by the        down-converting unit and obtain an image signal with the second        resolution for display.

Advantageous Effects of Invention

According to the present technology, it is possible to satisfactorilydetect an edge detection signal of a high frequency band from a capturedimage signal at all times. Note that the effects described in thepresent specification are merely examples, and not limitative; othereffects may be exhibited.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of a configuration ofan imaging device (a camera system) as a first embodiment.

FIG. 2 is a graph showing an example of frequency characteristics of ahigh-magnification (100 magnifying power) zoom lens.

FIG. 3 is a graph showing an example of characteristics of a highestfrequency band in accordance with F numbers.

FIG. 4 is a graph showing an example of frequency characteristics of ahigh-pass filter.

FIG. 5 is a graph for describing fluctuation of gains of an edgedetection signal.

FIG. 6 is a diagram illustrating an example of an edge informationdisplay of a viewfinder image (a down-converted image) of the imagingdevice.

FIG. 7 is a block diagram illustrating an example of a configuration ofa high frequency band edge detection circuit.

FIG. 8 is a block diagram illustrating an example of a configuration ofan imaging device (a camera system) as a second embodiment.

FIG. 9 is a block diagram illustrating an example of a configuration ofa high frequency band edge detection circuit.

FIG. 10 is a block diagram illustrating an example of a configuration ofan imaging device (a camera system) as a third embodiment.

FIG. 11 is a block diagram illustrating an example of a configuration ofa high frequency band edge detection circuit.

MODE(S) FOR CARRYING OUT THE INVENTION

Exemplary embodiments for implementing the invention (each of which willalso be referred to as an “embodiment” below) will be described below.Note that description will be provided in the following order.

1. First Embodiment

2. Second Embodiment

3. Third Embodiment

4. Modified example

1. First Embodiment

[Example of Configuration of Imaging Device]

FIG. 1 illustrates an example of a configuration of an imaging device (acamera system) 10 as a first embodiment. The imaging device 10 has acentral processing unit (CPU) 101, a lens unit 102, an imaging unit 103,an imaging system correction circuit 104, a gain adjustment circuit 105,a knee/gamma correction circuit 106, and an output signal generationcircuit 107 for main line output. In addition, the imaging device 10further has a down-converting circuit 108, an output signal generationcircuit 109 for a viewfinder, an edge detection circuit 110, and a highfrequency band edge detection circuit 111.

The CPU 101 constitutes a control unit and controls operations of theunits of the imaging device 10. The lens unit 102 concentrates lightfrom a subject, which is not illustrated, on an imaging sensor (an imagesensor) of the imaging unit 103. The imaging unit 103 is an imaging unitapplicable to 4K resolution. For example, while HD resolution has 1920pixels in the horizontal direction and 1080 pixels in the verticaldirection, 4K resolution has 3840 pixels in the horizontal direction and2160 pixels in the vertical direction. The imaging unit 103 receiveslight from the subject on the imaging sensor, performs photo-electricconversion and A/D conversion, and outputs a captured image signal.

The imaging system correction circuit 104 performs imaging systemcorrection on the captured image signal output from the imaging unit103, such as white balance correction, aberration correction, andshading correction. The gain adjustment circuit 105 performs gainadjustment on the captured image signal corrected by the imaging systemcorrection circuit 104. The knee/gamma correction circuit 106 performsknee correction for causing an output signal to comply with a signalstandard and gamma correction for handling monitor gamma on thegain-adjusted captured image signal. The output signal generationcircuit 107 converts the captured image signal that has been correctedby the knee/gamma correction circuit 106 to have a final output formatand outputs the image signal to an outside as main line output of a 4Kimage.

The down-converting circuit 108 performs a down-conversion process (aresolution conversion process) on the captured image signal corrected bythe knee/gamma correction circuit 106 and generates an image signal ofHD resolution. The output signal generation circuit 109 converts theimage signal of HD resolution obtained from the down-conversion processto have an output format that is suitable for the viewfinder (a displaydevice) on the output side and outputs the image signal to the outsideas viewfinder output of an HD image.

The edge detection circuit 110 obtains an edge detection signal with HDresolution on the basis of the image signal with HD resolution obtainedby the down-converting circuit 108. Specifically, the edge detectioncircuit 110 performs, for example, a high-pass filtering process on theimage signal with HD resolution for each pixel and calculates the edgedetection signal of HD resolution. The edge detection circuit 110constitutes a low-frequency band edge detection circuit.

The high frequency band edge detection circuit 111 obtains an edgedetection signal with HD resolution on the basis of the captured imagesignal with 4K resolution whose gain has been adjusted by the gainadjustment circuit 105. Specifically, the high frequency band edgedetection circuit 111 extracts the edge detection signal of a highfrequency band from the captured image signal with the 4K resolution,performs a down-conversion process on the edge detection signal, andthereby obtains an edge detection signal with HD resolution.

In this embodiment, the CPU 101 receives supply of lens information(e.g., information of a lens model number, a zoom position, an F number,and the like) from the lens unit 102. In addition, the CPU 101 receivessupply of imaging information (e.g., information of an exposure time, ashutter speed, and the like) from the imaging unit 103. Furthermore, theCPU 101 receives supply of correction information (e.g., information ofwhite balance correction and the like) from the imaging systemcorrection circuit 104. Moreover, the CPU 101 receives supply of gaininformation (a gain value) from the gain adjustment circuit 105.

In the embodiment, the CPU 101 controls the high frequency band edgedetection circuit 111 such that a pass-band of a filtering unit thereoffor extracting an edge detection signal is applicable to a frequency ofthe extracted edge detection signal on the basis of the lens information(the information of a lens model number, a zoom position, an F number,and the like).

That is, a frequency band of the captured image changes in accordancewith a zoom position. In a case in which a zoom lens is mounted toperform imaging with zooming, a frequency band of a captured imagemostly drops further in comparison to a case in which imaging isperformed at a uniform magnification in general.

A case in which, for example, a 4K camera with a high-magnification (a100-times magnification) zoom lens having lens frequency characteristicsas shown in FIG. 2 performs imaging may be considered. The solid line aindicates lens frequency characteristics when a zoom magnification is6.5 times, the dashed line b indicates lens frequency characteristicswhen a zoom magnification is 10 times, and the dashed line c indicateslens frequency characteristics when a zoom magnification is 29 times.

In this case, 4K resolution that is substantially close to a highestfrequency (around frequency of 1) can be obtained in the case in whichimaging is performed at a low magnification (zoom magnification of 6.5times). However, it is nearly not possible to obtain 4K resolution dueto the lens characteristics in the case in which imaging is performed ata high magnification (the zoom magnification of 29 times).

In a case in which a filter is designed such that a signal having anearly highest frequency is obtained at a uniform magnification, a highfrequency region is gradually lost when zooming is gradually performed,and thus it is difficult to obtain high frequency edge information in afiltering process. On the other hand, in a case in which a filter isdesigned such that a high frequency edge signal is obtained at a100-times magnification, excess edge signals are output at a uniformmagnification and thus it is difficult to determine a degree offocus-matching.

In addition, a frequency band of a captured image changes in accordancewith F numbers (aperture values). FIG. 3 shows an example ofcharacteristics of a highest frequency band in accordance with Fnumbers. In this case, the highest resolution appears around F5, and thegreater an F number is (the narrower an iris is), the lower theresolution is due to aperture blurring, but on the contrary, the smalleran F value is (the wider the iris is), the lower the resolution is dueto releasing blurring. Note that a frequency response in accordance withF numbers is uniquely determined depending on a lens model number (atype of lens).

In addition, a frequency band of a captured image changes in accordancewith lens model numbers (types of lenses). The lens here has MTFcharacteristics for its lens model number. The MTF characteristicsexpress spatial frequency characteristics of the lens.

In this embodiment, although details will be provided below, thefiltering unit of the high frequency band edge detection circuit 111 isconstituted by a first high-pass filter that has a high-band cutofffrequency, a second high-pass filter that has a low-band cutofffrequency, and an α blending unit that α-blends output of the high-passfilters.

As described above, the CPU 101 supplies filter factors e1 and e2 forconfiguring the first and second high-pass filters and an α value forα-blending to the high frequency band edge detection circuit 111 tocontrol a pass-band of the filtering unit of the high frequency bandedge detection circuit 111 for extracting an edge detection signal. Inthis case, the CPU 101 calculates, for example, the filer factors e1 ande2 on the basis of the lens model number and then calculates the α valueon the basis of the zoom position and the F number.

In addition, in this embodiment, the CPU 101 controls a gain of the edgedetection signal on the basis of the lens information (the informationof the lens model number, the zoom position, the F number, and the like)to make the gain of the edge detection signal extracted by the highfrequency band edge detection circuit 111 stabilized.

For example, the dashed line d of FIG. 4 indicates an example of afrequency characteristic of the first high-pass filter having ahigh-band cutoff frequency and the dashed-dotted line e of FIG. 4indicates an example of a frequency characteristic of the secondhigh-pass filter having a low-band cutoff frequency. As is shown in FIG.5 (a), a maximum gain in a case in which the low-band high-pass filteris applied (α=0) when a zoom magnification is set to 29 times isapproximately 0.3. Meanwhile, a maximum gain in a case in which thehigh-band high-pass filter is applied (α=1) when a zoom magnification isset to 6.5 times is approximately 0.2 as shown in FIG. 5 (b).

The CPU 101 supplies a gain factor g for make the gain of the edgedetection signal extracted by the filtering unit of the high frequencyband edge detection circuit 111 as described above uniform to the highfrequency band edge detection circuit 111. For example, in theabove-described example, the gain of the edge detection signal extractedby the filtering unit when the zoom magnification is set to 6.5 times is1.5 times the gain when the zoom magnification is set to 29 times. Inthis case, for example, the CPU 101 calculates the gain factor g fromthe above-described α value.

In addition, as will be described in detail in the embodiment, the highfrequency band edge detection circuit 111 includes a coring unit forreducing noise included in the edge detection signal extracted by thefiltering unit.

Coring is suppressing an input signal with amplitude that is smallerthan a certain value, regarding it as a noise component. It is knownthat an amount of random noise depends on a signal level (as signalamplitude becomes greater, noise increases accordingly). For thisreason, a coring level is generally changed in accordance with averageluminance of nearby input pixels.

In the embodiment, the CPU 101 modulates a coring level on the basis ofthe imaging information, the correction information, the gaininformation, and the like. In a case in which a gain-up process isperformed, for example, amplitude of random noise increases accordingly,and on the contrary, in a case in which a gain-down process isperformed, amplitude of random noise decreases accordingly. Thus, thecoring level is modulated in accordance with gains.

In the embodiment, the CPU 101 obtains a modulation factor m formodulating the coring level on the basis of the imaging information, thecorrection information, the gain information, and the like to modulatethe coring level and supplies the modulation factor to the highfrequency band edge detection circuit 111.

An operation of the imaging device 10 illustrated in FIG. 1 will bebriefly described. Light from a subject that has passed through the lensunit 102 is received by the imaging sensor (image sensor) of the imagingunit 103. Then, the imaging unit 103 performs photo-electric conversionand A/D conversion and thereby obtains a captured image signal with 4Kresolution. This captured image signal is appropriately processed by theimaging system correction circuit 104 and the gain adjustment circuit105 and then supplied to the knee/gamma correction circuit 106.

The knee/gamma correction circuit 106 performs knee correction forcausing an output signal to comply with a signal standard and gammacorrection for correcting monitor gamma on the captured image signal.The corrected captured image signal is supplied to the output signalgeneration circuit 107. The output signal generation circuit 107converts the captured image signal to have a final output format andoutputs the image signal to the outside as main line output for 4Kimages.

In addition, the captured image signal corrected by the knee/gammacorrection circuit 106 is supplied to the down-converting circuit 108.The down-converting circuit 108 performs a down-conversion process (aresolution conversion process) on the captured image signal and therebygenerates an image signal with HD resolution. This image signal with HDresolution is supplied to the output signal generation circuit 109. Theoutput signal generation circuit 109 converts the image signal with HDresolution to have an output format suitable for the viewfinder (displaydevice) on the output side and outputs the image signal to the outsideas viewfinder output for HD images.

An angle of view and focus of the main line output image is changed by auser through an operation of the lens unit 102 at any time. Since theviewfinder output corresponds to the main line output, the user cancheck a change in an angle of view and focus of the main line outputimage through the viewfinder output in real time.

In addition, the image signal with HD resolution obtained by thedown-converting circuit 108 is supplied to the edge detection circuit110. The edge detection circuit 110 performs, for example, a high-passfiltering process on the image signal with HD resolution for each ofpixels, and thereby obtains an edge detection signal EG_1 with HDresolution. This edge detection signal EG_1 with HD resolution issupplied to the output signal generation circuit 109 as a low-frequencyband edge detection signal.

In addition, the captured image signal with 4K resolution whose gain hasbeen adjusted by the gain adjustment circuit 104 is supplied to the highfrequency band edge detection circuit 111. The high frequency band edgedetection circuit 111 extracts an edge detection signal of a highfrequency band from the captured image signal with 4K resolution,performs a down-conversion process on the edge detection signal, andthereby obtains an edge detection signal EG_2 with HD resolution.

In this case, the CPU 101 controls the high frequency band edgedetection circuit 111 such that a pass-band of the filtering unit forextracting an edge detection signal is applicable to a frequency of theedge detection signal to be extracted on the basis of the lensinformation (the information of the lens model number, the zoomposition, the F number, and the like). To this end, the CPU 101 suppliesthe filter factors e1 and e2 for configuring the first and secondhigh-pass filters and the α value for α blending to the high frequencyband edge detection circuit 111.

Further, in this case, the CPU 101 controls a gain of the edge detectionsignal on the basis of the lens information (the information of the lensmodel number, the zoom position, the F number, and the like) to make thegain of the edge detection signal extracted by the high frequency bandedge detection circuit 111 stabilized. To this end, the CPU 101 suppliesthe gain factor g to the high frequency band edge detection circuit 111.

Moreover, in this case, the high frequency band edge detection circuit111 performs coring to suppress a noise component of the extracted edgedetection signal. The CPU 101 modulates a coring level on the basis ofthe imaging information, the correction information, the gaininformation, and the like. To this end, the CPU 101 supplies themodulation factor m to the high frequency band edge detection circuit111.

The output signal generation circuit 109 combines the image signal withHD resolution that would be the above-described viewfinder output withthe edge detection signal EG_1 of the low frequency band obtained by theedge detection circuit 110 and the edge detection signal EG_2 of thehigh frequency band obtained by the high frequency band edge detectioncircuit 111. In this case, edge information displays (e.g.,edge-emphasized displays) resulting from the combined edge detectionsignals EG_1 and EG_2 of both frequency bands are combined so as to bedistinguished by a difference, for example, a hue, luminance, or a typeof line (a solid line, a dashed line, or the like). Note that, in a casein which the edge information displays of both frequency bands aredistinguished by hues, it is desirable for the edge information displaysto be adjusted by the user so as to have easily-viewable colors orbrightness in accordance with color of the subject.

FIG. 6 illustrates an example of an edge information display (e.g., anedge-emphasized display) of the viewfinder image (a down-convertedimage) of the imaging device 10 illustrated in FIG. 1 . FIG. 6 (a)illustrates an example of an image of the image signal with 4Kresolution that would be the main line output. FIG. 6 (b) illustrates anexample of an image formed with the image signal with HD resolutionobtained by performing the down-conversion process on the image signalwith 4K resolution. FIG. 6 (c) illustrates an example of an imageobtained by adding only the edge detection signal EG_1 with HDresolution detected on the basis of the image signal with HD resolutionto the image signal with HD resolution. In this case, for example, lowfrequency band edges are highlighted in white.

FIG. 6 (d) illustrates an example of an image obtained by adding theedge detection signal EG_1 with HD resolution and the edge detectionsignal EG_2 with HD resolution, which has been obtained by performingthe filtering process on the captured image signal with 4K resolutionand further the down-conversion process thereon, to the image signalwith HD resolution. In this case, for example, low frequency band edgesare highlighted in white and high frequency band edges are highlightedin red.

When focus is to be adjusted on the basis of the viewfinder display, forexample, first, focus is roughly adjusted viewing the low frequency bandedges colored in white and then focus is finely adjusted, for example,with reference to the high frequency band edges colored in red.Accordingly, it is easy to accurately adjust focus for an image with 4Kresolution only with the viewfinder display of HD resolution.

[Example of Configuration of High Frequency Band Edge Detection Circuit]

FIG. 7 illustrates an example of a configuration of the high frequencyband edge detection circuit 111. The high frequency band edge detectioncircuit 111 processes the captured image in units of 2×2=4 pixels. Here,the 4 pixels are assumed to be a pixel In00, a pixel In01, a pixel In10,and a pixel In11. The high frequency band edge detection circuit 111 hasan In00 edge detection unit 121-0, an In01 edge detection unit 121-1, anIn10 edge detection unit 121-2, an In11 edge detection unit 121-3, adown-converting unit 122, a squaring unit 123, and a selector 124.

The In00 edge detection unit 121-0 obtains an edge detection signalcorresponding to the pixel In00 on the basis of signals of nearbypixels. The In01 edge detection unit 121-1 obtains an edge detectionsignal corresponding to the pixel In01 on the basis of signals of nearbypixels. The In10 edge detection unit 121-2 obtains an edge detectionsignal corresponding to the pixel In10 on the basis of signals of nearbypixels. The In11 edge detection unit 121-3 obtains an edge detectionsignal corresponding to the pixel In11 on the basis of signals of nearbypixels.

The In00 edge detection unit 121-0 has a horizontal edge detection unit125-h, a vertical edge detection unit 125-v, and a selector 126. Thehorizontal edge detection unit 125-h obtains a horizontal edge detectionsignal corresponding to the pixel In00 on the basis of signals of nearbypixels. The horizontal edge detection unit 125-h has a low-pass filter131, a high-pass filter (a first high-pass filter) 132, anotherhigh-pass filter (a second high-pass filter) 133, an α blending unit134, a coring unit 135, and a divider 136.

The high-pass filter 132 has the filter factor e1 supplied from the CPU101, has a high-band cutoff frequency (see the frequency characteristicindicated by the dashed line d of FIG. 4 ), and detects a horizontalsignal (an edge detection signal) of a high frequency band of the pixelIn00. The high-pass filter 133 has the filter factor e2 supplied fromthe CPU 101, has a low-band cutoff frequency (see the frequencycharacteristic indicated by the dashed line e of FIG. 4 ), and detects ahorizontal signal (an edge detection signal) of a high frequency band ofthe pixel In00. The α blending unit 134 performs α blending on output ofthe high-pass filters 132 and 133 using the α value and the gain factorg supplied from the CPU 101, further performs gain adjustment thereon,and thereby obtains an edge detection signal Edge.

The process of the α blending unit 134 can be expressed with thefollowing formula (1). Here, “HPF-H” represents output of the high-passfilter 132 and “HPF-L” represents output of the high-pass filter 133.The output of the high-pass filter 132 becomes more dominant as the αvalue increases, and on the contrary, the output of the high-pass filter133 becomes more dominant as the α value decreases.Edge={HPF−H×α+HPF-L+(1−α)}×g  (1)

The high-pass filters 132 and 133 and the α blending unit 134 constitutethe filtering unit for extracting an edge detection signal. Thepass-band of the filtering unit is controlled to deal with an edgedetection signal of a desired frequency band at all times with thefilter factors e1 and e2 and the α value supplied from the CPU 101. Forexample, α=0 when a maximum zoom-in is set, α=1 when no zooming is used,and α=0.5 when a zoom position is at the center. By flexibly changingthe α value in accordance with zoom positions in that manner, optimumoutput of the high-pass filters can be created no matter where the zoomposition is, and thus an edge detection signal of a desired frequencyband can be obtained at all times. In addition, since the CPU 101supplies the gain factor g, a gain of the edge detection signalextracted by the filtering unit can be controlled so as to be stable.

The low-pass filter 131 obtains an average value of horizontal signallevels of the nearby pixels of the pixel In00 when the filtering unitdetects the horizontal signal (an edge detection signal) of a highfrequency band of the pixel In00.

The coring unit 135 receives input of the signal (edge detection signal)of the high frequency band detected by the filtering unit and outputsthe signal with suppressed noise. Since the filtering unit responses notonly to edges but also to high frequency random noise, the signal (edgedetection signal) of the high frequency band detected by the filteringunit also includes the high frequency random noise. The coring unit 135suppresses an input signal with amplitude that is smaller than a certainvalue, regarding it as a noise component.

Specifically, if a level of the input signal is set to X and a coringlevel is set to CORE-LEVEL×m, a level of an output signal Y is obtainedby using the following formula (2). Here, “CORE-LEVEL” is an averagevalue of the signal levels of the nearby pixels obtained by the low-passfilter 131 and “m” is a modulation factor supplied from the CPU 101. Bymodulating “CORE-LEVEL” with the modulation factor m in that manner,noise components can be effectively reduced.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{Y = \left\{ \begin{matrix}{{X - {{CORE\_ LEVEL}*m}},} & {{if}\mspace{14mu}\left( {X \geq {{CORE\_ LEVEL}*m}} \right)} \\{0,} & {{if}\mspace{14mu}\left( {{{- {CORE\_ LEVEL}}*m} < X < {{CORE\_ LEVEL}*m}} \right)} \\{{X + {{CORE\_ LEVEL}*m}},} & {otherwise}\end{matrix} \right.} & (2)\end{matrix}$

The divider 136 divides the signal (edge detection signal) of the highfrequency band whose noise has been processed to be suppressed by thecoring unit 135 by the average value of the signal levels of the nearbypixels obtained by the low-pass filter 131 and normalizes the result.

In a case in which a natural image is imaged, in general, an amount ofspatial change increases as a signal level is higher and brighter, andthe amount of spatial change decreases as the signal level is lower anddarker. Thus, output of the filtering unit (output of the α blendingunit 134) easily has a high value in a bright region and a lower valuein a dark region. In other words, edges are easily erroneously detectedin bright regions and edges are difficult to detect in dark regions.

The divider 136 performs the normalization process to solve thatproblem. In this case, the value of the output of the filtering unit isnormalized to be a low value in a bright region in which output of thelow-pass filter 131 has a high value, and the value of the output of thefiltering unit is normalized to be a high value in a dark region inwhich output of the low-pass filter 131 has a low value. Thus, since thedivider 136 performs the normalization process, erroneous detection ofedges is suppressed in the bright region and edges are easily detectedin the dark region.

Although detailed description is omitted, the vertical edge detectionunit 125-v has a similar configuration to the above-described horizontaledge detection unit 125-h, and detects a vertical signal (an edgedetection signal) of a frequency band. The selector 126 selectivelytakes a signal having a greater absolute value between the horizontaledge detection signal of the pixel In00 detected by the horizontal edgedetection unit 125-h and the vertical edge detection signal of the pixelIn00 detected by the vertical edge detection unit 125-v, and outputs thesignal as an edge detection signal of the In00 edge detection unit121-0, i.e., an edge detection signal corresponding to the pixel In00.

Although detailed description is omitted, the In01 edge detection unit121-1, the In10 edge detection unit 121-2, and the In11 edge detectionunit 121-3 have a similar configuration to the above-described In00 edgedetection unit 121-0 and respectively output high frequency band signals(edge detection signals) corresponding to the pixel In01, the imageIn10, and the pixel In11.

The down-converting unit 122 performs a down-conversion process on theedge detection signals detected by the edge detection units 121-0,121-1, 121-2, and 121-3 and thereby obtains an edge detection signalwith HD resolution. The down-converting unit 122 performs thedown-conversion process from 4K resolution to HD resolution at a ratioof 2:1 in two horizontal and vertical directions.

In general, the down-conversion process is a process of thinning outpixels by an amount “the number of input pixels—the number of outputpixels” using any method. For example, down-conversion at the ratio of2:1 can be realized simply by thinning out every other pixel. However,edge information present in a thinning-out phase is lost during suchsimple thinning. Furthermore, generally in such a down-conversionprocess, band-limiting using a low-pass filter (LPF) is applied tosuppress folding distortion (aliasing) and then thinning is performed.If thinning is performed after the application of band-limiting,however, edge information in a high frequency band is lost due to thelow-band-limiting process.

For that reason, the down-converting unit 122 realizes thedown-conversion process from the 4K resolution to the HD resolution atthe ratio of 2:1 in the two horizontal and vertical directions whilehigh frequency band edges are maintained by having a difference betweencontinuous edge signals and devising to output a set of signals whosedifference is the maximum.

The squaring unit 123 makes the edge detection signal with HD resolutiongenerated by the down-converting unit 122 squared using a multiplier andoutputs the result. The gain obtained by the filtering unit can have aform of a quadratic function through the squaring process and thus thesignal of a narrower band range can be highlighted.

The selector 124 selectively outputs the linear edge detection signalfrom the down-converting unit 122 or the squared edge detection signalfrom the squaring unit 123, for example, on the basis of a selectionoperation by a cameraman (a user). Whether selecting the squared edgedetection signal and narrowing a range of a focus position in which highfrequency band edges are displayed or selecting the linear edgedetection signal and widening the range of the focus position in whichhigh frequency band edges are displayed makes focusing easier depends onpreference of the cameraman.

An operation of the high frequency band edge detection circuit 111illustrated in FIG. 7 will be briefly described. The horizontal edgedetection unit 125-h obtains a horizontal edge detection signal (a highfrequency band signal) of the pixel In00 on the basis of nearby pixelsfor every 4-pixel set. Likewise, the vertical edge detection unit 125-vobtains a vertical edge detection signal (a high frequency band signal)of the pixel In00 on the basis of the nearby pixels for every 4-pixelset.

The horizontal edge detection signal of the pixel In00 obtained by thehorizontal edge detection unit 125-h is supplied to the selector 126. Inaddition, the vertical edge detection signal of the pixel In00 obtainedby the vertical edge detection unit 125-v is supplied to the selector126. The selector 126 selectively takes a signal having a greaterabsolute value between the two edge detection signals and outputs thesignal as an edge detection signal of the In00 edge detection unit121-0. i.e., an edge detection signal corresponding to the pixel In00.

In addition, the In01 edge detection unit 121-1, the In10 edge detectionunit 121-2, and the In11 edge detection unit 121-3 respectively outputedge detection signals (high frequency band signals) corresponding tothe pixel In01, the image In10, and the pixel In11. Each of the edgedetection signals is supplied to the down-converting unit 122. Thedown-converting unit 122 performs the down-conversion process on theedge detection signals and thereby obtains an edge detection signal withHD resolution.

The edge detection signal with HD resolution obtained by thedown-converting unit 122 is supplied to the squaring unit 123. Thesquaring unit 123 makes the edge detection signal with HD resolutionsquared using the multiplier. The squared edge detection signal obtainedby the squaring unit 123 and the linear edge detection signal obtainedby the down-converting unit 122 are supplied to the selector 124.

The selector 124 selectively outputs the linear edge detection signal orthe squared edge detection signal, for example, on the basis of aselection operation by a cameraman (a user). This output is output ofthe high frequency band edge detection circuit 111. The high frequencyband edge detection circuit 111 obtains an edge detection signal at aratio of one output pixel “Out0” to four input pixels “In00, In01, In10,and In11.” That is, the edge detection signal EG_2 with HD resolution isobtained from the captured image signal with 4K resolution.

As described above, the imaging device 10 illustrated in FIG. 1 controlsthe high frequency band for extracting the edge detection signals fromthe captured image signal with 4K resolution, i.e., the pass-band of thefiltering unit constituted by the high-pass filters 132 and 133 and theα blending unit 134 included in the high frequency band edge detectioncircuit 111 on the basis of the lens information. Thus, even iffrequencies of the edge detection signals included in the captured imagesignal vary due to a change in a zoom position, a lens model number, anF number, and the like, the edge detection signals can be satisfactorilydetected at all times.

In addition, the imaging device 10 illustrated in FIG. 1 controls thegain of the edge detection signals extracted by the filtering unitconstituted by the high-pass filters 132 and 133 and the α blending unit134 included in the high frequency band edge detection circuit 111 onthe basis of the lens information. Thus, even if the gain of the edgedetection signals extracted by the filtering unit varies due to a changein the zoom position, the lens model number, the F number, and the like,the gain of the edge detection signals can be stabilized.

Furthermore, the imaging device 10 illustrated in FIG. 1 controls(modulates) a coring level on the basis of imaging information,correction information, and gain information when the coring unit 135reduces noise included in the edge detection signals extracted by thefiltering unit constituted by the high-pass filters 132 and 133 and theα blending unit 134 included in the high frequency band edge detectioncircuit 111. Thus, noise components included in the edge detectionsignals extracted by the filtering unit can be effectively reduced.

2. Second Embodiment

[Example of Configuration of Imaging Device]

FIG. 8 illustrates an example of a configuration of an imaging device (acamera system) 10A as a second embodiment. In FIG. 8 , the samereference numerals are given to constituent elements corresponding tothose of FIG. 1 and detailed description thereof is appropriatelyomitted.

The imaging device 10A has the same configuration as the imaging device10 illustrated in FIG. 1 except that the high frequency band edgedetection circuit 111 is replaced with the high frequency band edgedetection circuit 111A. The high frequency band edge detection circuit111 includes the filtering unit constituted by the high-pass filters 132and 133 and the α blending unit 134 to extract edge detection signals (ahigh frequency band signal) as described above (see FIG. 7 ).

The high frequency band edge detection circuit 111A of the imagingdevice 10A includes a filtering unit constituted by a band-pass filter137 as illustrated in FIG. 9. Constituent elements of FIG. 9corresponding to those of FIG. 7 are indicated by the same referencenumerals.

The CPU 101 controls the high frequency band edge detection circuit 111Asuch that a pass-band of the filtering unit for extracting edgedetection signals can be applicable to edge detection signals of adesired frequency band at all times on the basis of lens information(information including a lens model number, a zoom position, an Fnumber, and the like). To this end, the CPU 101 calculates a filterfactor e3 of the band-pass filter 137 on the basis of the lensinformation and supplies the filter factor e3 to the band-pass filter137.

Note that the band-pass filter 137 also performs a process forstabilizing a gain of an extracted edge detection signal (a highfrequency band signal). To this end, the CPU 101 calculates a gainfactor g on the basis of the lens information and supplies the gainfactor g to the band-pass filter 137.

As described above, the imaging device 10A illustrated in FIG. 8controls the pass-band of the filtering unit for extracting edgedetection signals from a captured image signal with 4K resolution on thebasis of the lens information, like the imaging device 10 illustrated inFIG. 1 , and therefore the same effect as that of the imaging device 10illustrated in FIG. 1 can be obtained.

3. Third Embodiment

[Example of Configuration of Imaging Device]

FIG. 10 illustrates an example of a configuration of an imaging device(a camera system) 10B as a third embodiment. In FIG. 10 , the samereference numerals are given to constituent elements corresponding tothose of FIG. 1 and detailed description thereof is appropriatelyomitted.

The imaging device 10B has the same configuration as the imaging device10 illustrated in FIG. 1 except that the high frequency band edgedetection circuit 111 is replaced with the high frequency band edgedetection circuit 111B. The high frequency band edge detection circuit111 includes the filtering unit constituted by the high-pass filters 132and 133 and the α blending unit 134 to extract edge detection signals(high frequency band signals) as described above (see FIG. 7 ).

The high frequency band edge detection circuit 111B of the imagingdevice 10B includes a filtering unit constituted by a fast Fouriertransform (FFT) circuit 138 and a frequency selection circuit 139 asillustrated in FIG. 11 . Constituent elements of FIG. 11 correspondingto those of FIG. 7 are indicated by the same reference numerals.

The fast Fourier transform (FFT) circuit 138 analyzes frequencies of acaptured image signal with 4K resolution. The frequency selectioncircuit 139 selectively extracts a frequency component of an edgedetection signal (a high frequency band signal) from frequencycomponents obtained through the frequency analysis by the fast Fouriertransform (FFT) circuit 138 and takes the edge detection signal.

The CPU 101 controls the high frequency band edge detection circuit 111Asuch that a pass-band of the filtering unit for extracting edgedetection signals can be applicable to edge detection signals of adesired frequency band at all times on the basis of lens information(information including a lens model number, a zoom position, an Fnumber, and the like). To this end, the CPU 101 calculates a targetfrequency ft that is a frequency to be extracted by the frequencyselection circuit 139 on the basis of the lens information and suppliesthe target frequency ft to the frequency selection circuit 139.

Note that the frequency selection circuit 139 also performs a processfor stabilizing a gain of an extracted edge detection signal (a highfrequency band signal). To this end, the CPU 101 calculates a gainfactor g on the basis of the lens information and supplies the gainfactor g to the frequency selection circuit 139.

As described above, the imaging device 10B illustrated in FIG. 10controls the pass-band of the filtering unit for extracting edgedetection signals from a captured image signal with 4K resolution on thebasis of the lens information, like the imaging device 10 illustrated inFIG. 1 , and therefore the same effect as that of the imaging device 10illustrated in FIG. 1 can be obtained.

4. Modified Example

Note that the example in which imaging resolution is 4K resolution anddisplay resolution of a viewfinder is HD resolution has been introducedin the above-described embodiments. Thus, the down-converting unit 122of the high frequency band edge detection circuit 111 performsdown-converting at the ratio of 2:1 in two horizontal and verticaldirections. In a case in which a ratio of imaging resolution and displayresolution of a viewfinder is N:1, for example, the high frequency bandedge detection circuit 111 performs down-converting a the ratio of N:1.

In addition, the example in which the viewfinder with HD resolutionperforms edge information display with HD resolution and edgeinformation display of two frequency band stages of edge informationdisplay of 4K resolution at the same time has been introduced in theabove-described embodiments. However, performing edge informationdisplay of three or more frequency band stages at the same time is alsoconsidered. In a case in which imaging resolution is 8K resolution anddisplay resolution of a viewfinder is HD resolution, for example, edgeinformation display with HD resolution, edge information display with 4Kresolution, and further edge information display of three frequency bandstages of edge information display with 8K resolution can be performedat the same time in a configuration similar to those of theabove-described embodiments.

Additionally, the present technology may also be configured as below.

(1)

An image signal processing device including:

a filtering unit configured to extract an edge detection signal of ahigh frequency band from an image signal obtained from imaging; and

a band control unit configured to control the high frequency band on thebasis of lens information.

(2)

The image signal processing device according to (1), in which thefiltering unit includes a first high-pass filter with a first cutofffrequency, a second high-pass filter with a second cutoff frequency thatis lower than the first cutoff frequency, and an α blending unit thatperforms α blending on output of the first high-pass filter and outputof the second high-pass filter.

(3)

The image signal processing device according to (2), in which the bandcontrol unit controls at least an α value of the α blending on the basisof the lens information.

(4)

The image signal processing device according to any of (1) to (3), inwhich the lens information includes at least zoom position information.

(5)

The image signal processing device according to any of (1) to (4),further including:

a gain control unit configured to control a gain of the edge detectionsignal extracted by the filtering unit on the basis of the lensinformation.

(6)

The image signal processing device according to any of (1) to (5),further including:

a coring unit configured to reduce noise included in the edge detectionsignal extracted by the filtering unit; and

a coring level control unit configured to control a coring level of thecoring unit on the basis of imaging information, correction information,and gain information.

(7)

An image signal processing method including:

a step of extracting an edge detection signal of a high frequency bandfrom an image signal obtained from imaging: and

a step of controlling the high frequency band on the basis of lensinformation.

(8)

An image signal processing device including:

a down-converting unit configured to perform a down-conversion processon an image signal with a first resolution obtained from imaging andgenerate an image signal with a second resolution that is lower than thefirst resolution;

a filtering unit configured to extract an edge detection signal of ahigh frequency band from the image signal with the first resolution;

a band control unit configured to control the high frequency band on thebasis of lens information;

a high frequency band edge detection unit configured to perform adown-conversion process on an edge detection signal extracted by thefiltering unit and obtain an edge detection signal with the secondresolution; and

a combining unit configured to combine the edge detection signalobtained by the high frequency band edge detection unit with the imagesignal with the second resolution generated by the down-converting unitand obtain an image signal with the second resolution for display.

(9)

The image signal processing device according to (8), in which thefiltering unit includes a first high-pass filter with a first cutofffrequency, a second high-pass filter with a second cutoff frequency thatis lower than the first cutoff frequency, and an α blending unit thatperforms α blending on output of the first high-pass filter and outputof the second high-pass filter.

(10)

The image signal processing device according to (9), in which the bandcontrol unit controls at least an α value of the α blending on the basisof the lens information.

The image signal processing device according to any of (8) to (10), inwhich the lens information includes at least zoom position information.

(12)

The image signal processing device according to any of (8) to (11), inwhich the first resolution is 4K resolution and the second resolution isHD resolution.

(13)

An image signal processing method including:

a step of performing a down-conversion process on an image signal with afirst resolution obtained from imaging and generating an image signalwith a second resolution that is lower than the first resolution;

a step of extracting an edge detection signal of a high frequency bandfrom the image signal with the first resolution;

a step of controlling the high frequency band on the basis of lensinformation;

a step of performing a down-conversion process on the extracted edgedetection signal and obtaining an edge detection signal with the secondresolution; and

a step of combining the obtained edge detection signal with thegenerated image signal with the second resolution and obtaining an imagesignal with the second resolution for display.

(14)

An imaging device including:

an imaging unit configured to obtain an imaging signal with a firstresolution; and

an image signal processing unit configured to process the image signalwith the first resolution obtained by the imaging unit and obtain animage signal with a second resolution for viewfinder display, which islower than the first resolution,

in which the image signal processing unit includes

-   -   a down-converting unit configured to perform a down-conversion        process on an image signal with the first resolution and        generate an image signal with the second resolution that is        lower than the first resolution,    -   a filtering unit configured to extract an edge detection signal        of a high frequency band from the image signal with the first        resolution,    -   a band control unit configured to control the high frequency        band on the basis of lens information,    -   a high frequency band edge detection unit configured to perform        a down-conversion process on an edge detection signal extracted        by the filtering unit and obtain an edge detection signal with        the second resolution, and    -   a combining unit configured to combine the edge detection signal        obtained by the high frequency band edge detection unit with the        image signal with the second resolution generated by the        down-converting unit and obtain an image signal with the second        resolution for display.

REFERENCE SIGNS LIST

-   10, 10A, 10B imaging device (camera system)-   101 CPU-   102 lens unit-   103 imaging unit-   104 imaging system correction circuit-   105 gain adjustment circuit-   106 knee/gamma correction circuit-   107 output signal generation circuit (main line output)-   108 down-converting circuit-   109 output signal generation circuit (finder output)-   110 edge detection circuit-   111, 111A, 111B high frequency band edge detection circuit-   121-0 In00 edge detection unit-   121-1 In01 edge detection unit-   121-2 In10 edge detection unit-   121-3 In11 edge detection unit-   122 down-converting unit-   123 squaring circuit-   124 selector-   125-h horizontal edge detection unit-   125-v vertical edge detection unit-   126 selector-   131 low-pass filter-   132, 133 high-pass filter-   134 α blending unit-   135 coring unit-   136 divider-   137 band-pass filter-   138 fast Fourier transform circuit-   139 frequency selection circuit

The invention claimed is:
 1. An image processing device, comprising:circuitry configured to generate filter coefficients used to extractedges in a high frequency band from a first image signal that has afirst resolution and that is obtained from an imaging device, the filtercoefficients being generated using lens information obtained from a lensunit and information used at a time of imaging, generate first edgeinformation having a second resolution by down-converting an edgeinformation having the first resolution, the edge information having thefirst resolution being generated from the first image signal by a firstfiltering process based on the filter coefficients, the first resolutionbeing higher than the second resolution, generate second edgeinformation having the second resolution from a second image signal by asecond filtering process, the second image signal being generated bydown-converting the first image signal having the first resolution tothe first image signal having the second resolution, generate third edgeinformation for display by combining the first edge information with thesecond edge information, the third edge information being configured tovisually distinguish the first edge information from the second edgeinformation, and cause the third edge information to be displayed on adisplay by overlaying the third edge information on the second image. 2.The image processing device according to claim 1, wherein the display isa view finder.
 3. The image processing device according to claim 1,wherein a color and brightness of each of the first, second, and thirdedge information are user adjustable.
 4. The image processing deviceaccording to claim 1, wherein the lens information includes at least oneof type number, zoom position, and F value.
 5. The image processingdevice according to claim 1, wherein the information used at the time ofimaging includes at least one of imaging parameters, correctionparameters, and gain adjustment parameters.
 6. The image processingdevice according to claim 1, wherein the filter coefficients includecoefficients for α blending and high-pass filtering.
 7. The imageprocessing device according to claim 1, wherein the filter coefficientsinclude coefficients for band-pass filtering.
 8. The image processingdevice according to claim 1, wherein the filter coefficients includesFast Fourier Transform (FFT) coefficients.
 9. The image processingdevice according to claim 1, wherein the first filtering processincludes high-pass filtering the first image signal.
 10. The imageprocessing device according to claim 1, wherein the first filteringprocess includes bandpass filtering the first image signal.
 11. Theimage processing device according to claim 1, wherein the firstfiltering process includes performing a Fast Fourier Transform (FFT) onthe first image signal.
 12. The image processing device according toclaim 1, wherein the first resolution is 4K resolution and the secondresolution is HD resolution.
 13. The image processing device accordingto claim 1, wherein the first resolution is 8K.
 14. The image signalprocessing device according to claim 1, wherein the first resolutionincludes more than 3840 pixels in a horizontal direction and more than2160 pixels in a vertical direction.
 15. The image signal processingdevice according to claim 2, wherein the circuitry is further configuredto: reduce noise included in the first and second edge information, andcontrol a coring level on the basis of imaging information, correctioninformation, and gain information.
 16. An image signal processingmethod, comprising: generating, with circuitry, filter coefficients usedto extract edges in a high frequency band from a first image signal thathas a first resolution and that is obtained from an imaging device, thefilter coefficients being generated using lens information obtained froma lens unit and information used at a time of imaging; generating, withthe circuitry, first edge information having a second resolution bydown-converting an edge information having the first resolution, theedge information having the first resolution being generated from thefirst image signal by a first filtering process based on the filtercoefficients, the first resolution being higher than the secondresolution; generating, with the circuitry, second edge informationhaving the second resolution from a second image signal by a secondfiltering process, the second image signal being generated bydown-converting the first image signal having the first resolution tothe first image signal having the second resolution; generating, withthe circuitry, third edge information for display by combining the firstedge information with the second edge information, the third edgeinformation being configured to visually distinguish the first edgeinformation from the second edge information; and causing, with thecircuitry, the third edge information to be displayed on a display byoverlaying the third edge information on the second image.
 17. Anon-transitory computer-readable medium encoded with computer-readableinstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform an image processing method, comprising:generating filter coefficients used to extract edges in a high frequencyband from a first image signal that has a first resolution and that isobtained from an imaging device, the filter coefficients being generatedusing lens information obtained from a lens unit and information used ata time of imaging; generating first edge information having a secondresolution by down-converting an edge information having the firstresolution, the edge information having the first resolution beinggenerated from the first image signal by a first filtering process basedon the filter coefficients, the first resolution being higher than thesecond resolution; generating second edge information having the secondresolution from a second image signal by a second filtering process, thesecond image signal being generated by down-converting the first imagesignal having the first resolution to the first image signal having thesecond resolution; generating third edge information for display bycombining the first edge information with the second edge information,the third edge information being configured to visually distinguish thefirst edge information from the second edge information; and causing thethird edge information to be displayed on a display by overlaying thethird edge information on the second image.